Software Magazine - February/March 2001

Software Magazine - February/March 2001

Collaboration: Development & Management
Collaborating on Project Success

Collaborative management techniques bridge the needs of developers, project managers, and infrastructure operations staff to ensure the applications needed to support e-business can be built quickly and managed effectively. Here is a report on trends in collaborative management to support e-business from The Standish Group.

By Jim Johnson, Karen D. Boucher, Kyle Connors, and James Robinson

2001-02 Collaborative Management

Project Management:
The Criteria for Success

There's great news on the project management front, according to The Standish Group, West Yarmouth, Mass., a research firm that focuses on mission-critical project management applications. Its most recent findings indicate researchers and project managers alike are learning how to become more successful at IT project management. According to results of Standish's 1994 CHAOS study, a research report published annually since that year, only 28,000 application development projects met the criteria for success—completed on time, on budget, and with all the features and functions originally specified. Last year's results show that 78,000 U.S. projects were successful.

The reasons for the increase in successful projects vary. First, the average cost of a project has been more than cut in half. Better tools have been created to monitor and control progress, and more highly skilled project managers are using improved management processes. The fact that there are processes is significant in itself.

Most of these new projects are well within The Standish Group's criteria established in "Recipe for Success, 1998," which limits project duration to six months and project staff to six people. This article is based on information from the company's latest paper, "Extreme CHAOS 2001."

The Road to success

Success rates are up across the board, while cost and schedule overruns are declining. The CHAOS research timeline is evidence of steady improvement in IT project management. In 1994, only 16% of application development projects met the criteria for success—completed on time, on budget, and with all features/functions originally specified. In 2000, 28% of projects were in the successful column. (See the figure below.)

Standish categorizes projects into three resolution types:

Successful: The project is completed on time and on budget, with all features and functions originally specified.

Challenged: The project is completed and operational, but overbudget, late, and with fewer features and functions than initially specified.

Failed: The project is canceled before completion, or never implemented.

Tracking U.S. project outcomes showed that in 1994, 28,000 projects were successful, while in 2000, the number rose to 78,000—almost a threefold increase. Conversely, failed projects amounted to 54,000 in the 1994 study vs. 65,000 in the 2000 study. This was an 18% increase, while overall project growth exceeded 60%. Challenged projects grew at a rate of 62%, to equal 137,000 over the 1994 number of 93,000.

2001-02 Projects Show Improvement

Cost overruns in 1994 equaled 189% over the original estimate. This was reduced from 69% in the 1998 study and down to 45% in the 2000 study. Time overruns dropped from 222% in 1994 to 63% in 2000. Another piece of good news is that in 1994 only 61% of the required features were delivered on challenged projects, compared with 67% in the 2000 study.

Overall, the outlook is good. Project success rates are up, and overruns are down. More importantly, although the number of projects is expected to double in 2001, the rate of failure is expected to decline substantially.

What ensures a project's success? The original CHAOS study, conducted in 1994, identified 10 success factors. Standish has updated the CHAOS Ten for 2000. Although no project requires all 10 factors to be successful, the more factors present in the project strategy, the higher the confidence level. (See Table 1, "Recipe for Success: CHAOS Ten," below.)


	Recipe for Success: CHAOS Ten

	Confidence Level	Success Factors
	Executive support	18
	User involvement	16
	Experienced project manager	14
	Clear business objectives	12
	Minimized scope	10
	Standard software infrastructure	8
	Firm basic requirements	6
	Formal methodology	6
	Reliable estimates	5
	Other criteria	5

Table 1. Each factor has been weighted according to its influence on project success. The more points, the lower the project risk.

1 Executive support. Traditionally, executive support occupied the No. 2 spot; however, it is now the No. 1 factor in project failure. Executive support influences a project's process and progress. Lack of executive input can jeopardize a project.

2 User involvement. Lack of user involvement traditionally has been the No. 1 reason for project failure. Conversely, it has been the leading contributor to project success. Even when delivered on time and on budget, a project can fail if it doesn't meet user needs or expectations. However, this year user involvement has moved to the No. 2 position. Despite how that may sound, user involvement hasn't decreased in importance; it's just that IT professionals have, in effect, solved this major problem.

3 Experienced project manager. Ninety-seven percent of successful projects have an experienced project manager at the helm.

4 Clear business objectives. This factor has moved down one spot because evidence shows that experienced project managers increase success rates.

5 Minimized scope. Wrapping up the top five is minimized scope. Time is the enemy of all projects, and since scope affects time, or project duration, they are linked. Clearly then, minimizing scope increases a project's chances of success. Minimized scope has replaced small milestones. While these two factors are similar, the act of minimizing scope leads to greater success than does creating small milestones. Concentrating on the top five will result in 70 success points.

6 Standard software infrastructure. Requirements are in a state of constant flux, but infrastructure needs stability. The Standish Group's research shows that 70% of application code is infrastructure. Some of this code is unique to the application; nonetheless, much of this code could be purchased from an infrastructure vendor.

By using standard infrastructure, the application development team can concentrate on business rules rather than on technology. Many application development projects fail not in stand-alone application development, but in existing application integration. Standard infrastructures can shortcut application integration.

7 Firm basic requirements. The word "basic" refers to base-level requirements. Creating minimal, obtainable base requirements and then developing those features will reduce the effect of change. Delivering minimal features allows users and executive sponsors to see quick results. As a result, project managers are better prepared to articulate the needs and priorities of the next project phase.

8 Formal methodology. This provides a realistic picture of the project and resources committed to it. And it results in steps and procedures the team can reproduce and reuse. It also enables the team to maximize consistency. And it incorporates lessons learned into active projects. The process encourages a go or no-go decision checkpoint. It also helps the project team proceed with a higher level of confidence, or halt or alter steps to fit changing requirements. CHAOS research shows that 46% of successful projects use a formal project management methodology, compared with 30% of challenged and failed projects. So, this factor should increase success rates by about 16%.

9 Reliable estimates. Systematic project estimating must be approached realistically, because estimating is just plain hard. Then add to that the difficulty of developing, purchasing, and integrating components into existing and packaged applications, and outside services. IT managers must use all their collective knowledge and experience to come up with estimates that reflect the true effort required.

10 Other criteria. In last place is a collection of other factors. These factors include small milestones, proper planning, competent staff, and ownership. In the past, each of these factors was given its own category.

The CHAOS Ten success factors continue to be valuable for assessing project potential. While nothing can guarantee project success, adhering to the CHAOS Ten will increase your odds of putting together a winning project.

The Standish Group predicts that most projects started and resolved this year will be microprojects: ones lasting no more than four months and run by four people. CHAOS research shows minimization as a key factor in successful projects. The microproject is the ultimate in minimization. Many last only three to four weeks. Don't confuse microprojects with milestones—microprojects live and die on usable deliverables.

The Web and standard infrastructure have made the microproject a viable entity. New application versions can be brought up quickly, and bugs can be found, corrected online, and benefits realized immediately. There's no need for release dates, shipping codes, or drawn-out user training. One of Standish's Fortune 500 clients mandated microprojects. The company's president must approve any project estimated to cost more than $100,000.

However, the advent of the microproject has made it harder to manage resources, and has turned code version management into a nightmare. While microprojects are deliverables by themselves, they rarely stand alone. Therefore, changes in one project can affect the collection of objects, individual programs, interfaces, and databases. These entities could be in the thousands, while teams and developers span time zones. To address these issues requires an asset-based change management tool.

An asset-based tool that supports configuration management over multiple technologies and programming styles is just the ante. The tool must support versions and relationships of source files, directories, test cases, and configurations. The tool must also be able to communicate the project to the development community to prevent duplication and make developers aware of the project's use and reuse. It should control concurrent and parallel versions and use roles-based access control. This type of tool can go a long way in maintaining and developing successful applications.

Collaborating on Project Success - Part II

PSA vs. EPM

Two types of project management tools help service groups in this regard: professional service automation (PSA) and enterprise project management (EPM).

PSA tools form a suite of software modules or applications that together handle multiple projects and contributors (e.g., staff and contractors). PSA focuses on optimizing service processes for acquiring, managing, and fulfilling service engagements external to the enterprise. EPM software modules also manage multiple projects. However, EPM tools are most often used to manage the enterprise organization's internal projects.

Although similar, PSA and EPM tools aren't identical products with different names. Since both share functions and applications, however, is it possible or useful for project management to differentiate between the two when choosing vendors? The answer is a qualified yes. Rather than pigeonholing vendor products into one category or the other, project managers can benefit from differentiating the tools based on their functional modules.

Professional service organizations (PSOs) have several mission-critical needs that may be secondary or less important to enterprise project organizations (EPOs). Several of these mission-critical PSA needs are as follows:

Up-to-date and current inventory, staff-resource allocation, and management;
Contract life-cycle and fulfillment management;
Time and materials management;
Prospect marketing and business development management;
Profitability measurement and tracking;
Identifying and managing reusable best practices.

PSA software should support PSOs in identifying profitable business opportunities, entering into profitable contracts, and expediting and fulfilling those contracts with a high level of customer satisfaction.

IT groups looking to use EPMs may or may not have those same needs. It depends on factors that vary across enterprises:

Number, size, and complexity of IT service projects;
IT group engagement and contractual process for servicing internal groups;
The nature of IT's service "charter" implicitly, or explicitly (in writing), including internal politics;
Existing budgeting/sponsorship processes for IT services.

Differing functional modules or applications can be bundled with either PSAs or EPMs. (Vendors use their own proprietary names for describing the same functional components. Descriptions here address PSA and EPM functional characteristics in generic terms.) The following overview can shed further light on some subtle differences and similarities between PSAs and EPMs. During the vendor selection process, IT buyers can use these as checklists.

Resource manager modules are the heart of most PSA tools. They are used to staff engagements based on available resource skill, and pinpoint the best talent to perform required tasks. A project's skill requirements are inputted and compared with the staff skills inventory. Work plans matching staff availability to skill requirements are then created. Resource managers use a common repository of staff skills and historical project information.

The engagement manager tracks and reports on key milestones and dependencies during the service engagement. Project revenues, expenses, and resource schedules as well as task completion status can be viewed, analyzed, and modified as necessary. In addition to identifying missed milestones, the engagement manager can flag potential problems and issues. The engagement manager facilitates addressing problems and issues with appropriate management at the right time.

The knowledge manager module helps project personnel understand, share, and reuse knowledge and experiences gained from other service engagements. This module may help identify best practices for reuse or fill in staff on actions worth avoiding in the future. A knowledge manager module can also help users create and track staff development plans.

The project manager module schedules, tracks, and reports on individual projects or a project portfolio. In general, the project manager application reports time and progress, showing how they stack up against assigned tasks and work breakdowns, thus providing the real picture of project performance. Views such as Pert and Gantt charts offer easy interpretation. EPM and PM tools differ most in that EPM's common database or repository gives project stakeholders and other applications access to all project information.

A content manager publishes project progress to stakeholders internally (for EPM) or to clients externally (PSA). Content manager functionality should not be confused with Web content managers. Project administrators and team members use content managers to publish project status, schedules, summaries, and key project deliverables. Some high-end content managers use agent technology to automatically publish project status and summaries. Others provide problem and issue notification similar to the engagement manager. The content manager's appropriate detail and nature have become a subject for lively debate in PSO and EPO organizations.

The following are unique to PSAs: The marketing manager can forecast potential sales engagements and balance these against resource requirements, helping marketing management make go or no-go decisions on the details of service engagements. Projects are selected from a prospect portfolio by analyzing past margins or considering currently available skills. It also can be helpful in proposal writing and sales pipeline tracking. Typically, enterprise service organizations (ESOs) do not need this module's depth of functionality, since more often than not they are already chartered and funded to service their internal clients. To ESOs, quality of service is more important than profitability and number of service engagements. Of course, a pipeline of qualified prospects that develop into profitable engagements is critical to PSO success.

The financial manager function handles such tasks as the timely administration of contracts and client billing. This application can account for time and material, fixed price, performance-based, and payment schedule contracts. It also can be used for revenue recognition across clients, projects, geographies, staff, and partners. Again, because ESOs tend to be measured on the basis of their quality of service and not their profitability, this module is primarily a PSA feature.

The following are unique to EPMs: A comprehensive plan manager differentiates EPM suites from project management tools. This module provides tools for creating and updating project plans. After inputting different project parameters, users can examine what-if scenarios, with time and resource needs projected. The repository or central database is updated with the modified plan. IT service management can then track and further modify the plan as required. In the case of external service engagements supported by PSAs, modifying an original service contract with clients is usually done on an exception basis only.

A capacity manager allocates available resources across multiple projects. Although this function is in some ways very similar to a PSA's resource manager, it doesn't generate a plan based on resources. In an enterprise, the project office or IT service management is expected to resolve any allocation conflicts. Therefore, service management basically uses this module to forecast availability of resources for specific skills. Some products allow for multiple views. In addition, this module can facilitate granting and denying allocation requests.

Ultimately, PSA solutions and EPM software both enable service organizations to manage multiple projects, track time and tasks, publish results, and manage skills. The bottom line for service organizations is that both can help in managing the three pillars of project management success: time, money, and function.

It's the Infrastructure, Stupid

Webster's defines infrastructure as the underlying foundation or basic framework (of a system or an organization). When CHAOS University attendees were asked what this word meant, they said that an infrastructure includes a whole host of things that must be managed—from telephone systems and hardware rooms to personal productivity systems sitting on desktops. Infrastructure is everything that ensures an application can run.

When researchers talk about the importance of infrastructure for project completion, they narrow that definition to software. Software infrastructure is the heart, lungs, and circulatory system that jump-starts the application and keeps it running. Standish's research shows that a software application is, on average, 70% infrastructure code. This code isn't directly tied to the project's business requirements; rather, it provides connectivity, security, availability, integrity, and so on.

Products that provide infrastructure services fall under the middleware label and range from those offering simple data connectivity to ones with a whole portfolio of application services. The Standish Group believes it is imperative that project teams know about commercially available products that provide these functions and that they take advantage of them. Standish's latest research shows that projects that create middleware have no chance of being completed on time and on budget!

Key to this issue is the question, New or redeveloped applications most often include which types of services? Standish found the answer in its most recent International Demand Assessment and Requirements Tracking Study (I-DARTS), by reviewing 183 applications and asking participants, "What type of middleware functions will this application use?" (See Table 2, "Middleware Functions" for the results, below.)


	Middleware Functions

	Types	Percentage Used
	App-to-app messaging	49
	Transactional messaging	44
	Stored procedures	42
	Auto restart/failover	32
	Asynchronous messaging	29
	Message queuing	29
	2PC	27
	Remote Procedure Call	23
	Broadcasting	20
	Guaranteed delivery	19
	Publish/subscribe	9

Table 2. Shown here are the types of middleware functions required for new or extensively redeveloped applications.

Connectivity involves more than system and data integration; it includes integration with other applications. The number of specialized systems required to grow a business and help it stay competitive has exploded over the past few years. Today, a typical business may have customer relationship management, financials, personnel, and so on. Each of these systems grows into an island of data. Each island develops a culture and style of its own. Some island cultures speak in integers, some in fractions. Some are very careful as to what data they will accept (in terms of content and format), while others aren't quite so discriminating. Application integration solutions bridge these islands.

All these application links create a maze of integration points. And any weak link could lead to chaos. With so many software projects requiring this type of infrastructure service, it's important to be able to select the right solution for each application.

Integration approaches range from the classic export/transfer/load (ETL) operation, to real-time, event-driven sharing of information among applications. ETL is usually executed on some predetermined schedule and moves a snapshot of data from one system to another. The result is that, for some period of time, one part of the system doesn't have current information. This can usually be overcome by scheduling other tasks accordingly (e.g., cutting paychecks after validating new employees). ETL is the most common and widely used approach for sharing information. By contrast, event-driven integration moves data between systems as soon as it becomes available, keeping the entire system up-to-date on a near real-time basis.

Integration bridges come in many styles and are often distinguished by the way applications communicate. In Table 2, application-to-application messaging, transactional messaging, asynchronous messaging, message queuing, Remote Procedure Calls, broadcasting, guaranteed delivery, and publish/subscribe are all forms of bridges.

In addition to these integration options, a company should consider other services their application may require. These could include security, integrity, and availability. Sometimes the range of projects can be daunting—leading to the common reaction of wanting to just develop something in-house. However, the risky nature of doing so can't be stressed enough.

When it comes to software infrastructure, the most common reason for failure is relying on custom code to develop services. As Standish research shows, this approach always leads to time and cost delays. The market offers technology proven to ensure a solid infrastructure. Why not take advantage of it?

For years The Standish Group has focused on how and why development projects succeed or fail to identify methodologies involved. Traditional project management entails delegating assignments, allocating resources, providing time and cost estimates, and monitoring staff performance of management assignments. At the helm is the project manager who plays the pivotal role as chief coordinator and communicator. Project managers serve as the project focal point. The Standish Group has identified the project manager's key characteristics: the ability to translate business and technical requirements between the people from these respective disciplines; the competency to decrease project scope, thereby reducing the time frame; the ability to organize all participants; the ingenuity to provide direction, motivation, and inspiration; and the ability to clearly convey project requirements and progress.

Collaboration management has a twofold benefit: It provides an open channel of communication that gives everyone a voice, and when key personnel leave, the project is not completely dissolved. In this setting, project details are distributed effectively, and activities are self-coordinated, expedited, and documented.

Jim Johnson (jim@standishgroup.com) is chairman of The Standish Group, West Yarmouth, Mass. Karen D. Boucher (karen@standishgroup.com) is executive vice president. Kyle Connors (kyle@standishgroup.com) is a research adviser at Standish. James Robinson (james@standishgroup.com) is also a research adviser. Call The Standish Group at 508-760-3600 or visit www.standishgroup.com.